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Abstract — In this paper, we study joint network coding and 
distributed source coding of inter-node dependent messages, witli 
tlie perspective of compressed sensing. Specifically, the theoretical 
guarantees for robust fi-min recovery of an under-determined 
set of linear network coded sparse messages are investigated. We 
discuss the guarantees for ^i-min decoding of quantized network 
coded messages, based on Restricted Isometry Property (RIP) of 
the resulting measurement matrix. This is done by deriving the 
relation between tail probability of ^2 -norms and satisfaction of 
RIP. The obtained relation is then used to compare our designed 
measurement matrix, with i.i.d. Gaussian measurement matrix, 
in terms of RIP satisfaction. Finally, we present our numerical 
evaluations, which shows that the proposed design of network 
coding coefficients results in a measurement matrix with an RIP 
behavior, similar to that of i.i.d. Gaussian matrix. 

Index Terms — Compressed sensing, linear network coding, re- 
stricted isometry property, ^i-min decoding, Gaussian ensembles. 



I. Introduction 

Efficient data gathering in sensor networks has been the 
topic of many research projects where different applications 
have been considered. One of the concerns in data gathering is 
to take care of inter-node redundancy during the transmission. 
When the knowledge of inter-node dependency is known at 
the encoders (i.e. sensor nodes) and the decoder node(s), 
distributed source coding iT^ and optimal packet forwarding is 
the best transmission method, in terms of achieved informa- 
tion rates pi. However, flexibility and robustness to network 
changes, and no need (et the encoders) to the knowledge of 
inter-node dependency has drawn attention to random linear 
network coding |3| as an alternative transmission method |4j. 

Recently, the concepts of compressed sensing |5| have been 
used to perform an embedded distributed source coding in 
linear network coding of correlated or sparse messages (|6)- 
|9). Joint source, channel, and network coding is studied in 
1 8), Q, where analogue network coding |10| is used as a 
linear mapping to decrease temporal and spatial redundancy of 
sensor data. In yjj, we proposed Quantized Network Coding 
(QNC) with €i-min decoding , where the sparse messages can 
be recovered from smaller number of packets compared to the 
conventional linear network coding |3|. 

To guarantee robust £i-min recovery of messages from 
an under-determined set of linear measurements, the total 
measurement matrix has to be appropriate (or in other words 
satisfy some special properties). For instance, if it satisfies 
Restricted Isometry Property (RIP) of appropriate order, then 
^i-min recovery is feasible |12j, |13J. However, the literature 



of compressed sensing-based network coding does not include 
any result discussing theoretical (or even practical) require- 
ments for robust i?i-min recovery of linear network coded 
messages. 

In this paper, we discuss theoretical guarantees for ^i-min 
decoding of quantized network coded messages, based on 
RIP. Specifically, we discuss the satisfaction of RIP and its 
implications for the measurement matrix, resulting from the 
design of local network coding coefficients, proposed in fTTl. 

The description of data gathering scenario and formulation 
of our proposed quantized network coding [llj is presented 
in section III] This is followed by a discussion on choosing 
appropriate local network coding coefficients, which result 
in zero mean Gaussian entries for the measurement matrix, 
in section [III] In section |IV| we derive the relation between 
the tail probability of ^2-norms and satisfaction of RIP, and 
discuss satisfaction of RIP for our designed measurement 
matrices. In section W\ a numerical example is presented, 
which compares the measurement matrix, resulting from our 
QNC scenario with the case of perfect Gaussian measurement 
matrix. Finally, in section |VI] we discuss our concluding 
remarks on satisfaction of RIP in our QNC scenario. 

II. Quantized Network Coding with £i-min 
Decoding in Lossless Networks 

In this paper, we consider a lossless sensor network, repre- 
sented by a directed graph, Q ~ (V, £), where V = {1, . . . , n} 
is the set of its nodes. £ ~ {!,..., \£\] is also the set of 
edges (links), where each edge, e £ £, maintains a lossless 
communication from tail{e) node to head{e) node, at a 
maximum rate of Ce bits per link use. As a result, the input 
content of edge e at time t, represented by j/e(i) (since the 
links are lossless, input and output contents of each edge are 
the same), is from a discrete finite alphabet of size 2^"^^. 
Time index, t, is integer and a time unit represents the time in 
which blocks of length L are transmitted over all edges. The 
sets of incoming and outgoing edges of node v, are defined 
respectively: 

In{v) ~ {e : head{e) — v,e £ £}, 
Out{v) — {e : tail{e) = f , e G £}. 

We assume that each node v has a random information source, 
Xy, which generates (random) message, called x^, where Xy G 
M. Furthermore, we consider the case where the messages, 
x—[x.i,:v£V]g M", are such that there is a linear transform 



matrix. 



for which x 



s, and s is fc-sparse (has at 



most k non-zero elements). In (single session) data gathering, 
all the messages, Xy's, are to be transmitted to a single gateway 
(or decoder) node, represented by vq, where vq e V. 

QNC at each node, u e V, was defined in 1 1 1 1, as follows: 



2/. 



W=Qe[ Yl l3e.e'{t)-yeit-l)+a,^,{t)-X, 



(1) 



e' GIn{v) 



where Qe[.] is the quantizer (designed based on the value of 
Ce and the distribution of incoming contents and messages), 
associated with the outgoing edge e e Out{v), and Pe.e'{t) 
and ae,t,(t) are the corresponding network coding coefficients, 
picked from real numbers. Initial rest condition is also assumed 
to be satisfied in our QNC scenario: 2/e(l) = 0, V e £ £. 
We represent the quantization error of Qe[.] by 71^(1), which 
implies: 

ye{t)^ Y, Pe,e'{t)-ye{t-l)+ae,^(t)-X^+ne{t). (2) 
e' GIn{v) 

Equivalently, we have pT[ : 

yit) = Fit) ■ yit - 1) + A{t) ■ x + n(i), (3) 

where y{t) = [ye{t) : e E £], n{t) = [ne{t) : e E £], and, 

/3e.e'(t) , tail{e) — head{e') 



Fit) 



\£\x\£\ 



{F{t)}e,e' = 







^W|£|x|V| : {Mt)}e,v 



, Otherwise 

ae.v{t) , tail{e) = v 
, otherwise 



By using linearity in the QNC scenario, the marginal mea- 
surements at time t, represented by {z(i)}i's, where z{t) = 
[ye{t) ■ e E In{vo)], are calculated as: 

z{t) = B{t) ■ yit) = *(t) • X + n,ffit). (4) 

In Eq. [4] ^(t) and n^sJt) are defined as: 

t t'+i 

^(t) = i?(i)-^n^(^")-^(o, (5) 

t'=2 t"=t 
t t' + l 

n^ffit) = i?(i)-^n^(^")"^(*')' (6) 

t'=2 t"=t 

and B{t) is defined such that: 

bi^e{t) , i corresponds to e, e E In{vo) 



{B{t)h,e = 







otherwise 



We store marginal measurements, at the decoder, and build 
up total measurements vector, called z^^^it), as follows: 



Ztotit) 



m 
m 



where 



m = {t - l)\In{vo)l 
and for which we have | fTT| : 

Ztotit) = 'i'totit) -X + n^ff^t^tit), 



(7) 

(8) 
(9) 



where the total measurement matrix, '^fot{t), and total effec- 
tive noise vector, n^n ^^^{t), are calculated as follows: 



■^tot{t) = 



*(2) 



*(t) 



^eff,tot 



(i) = 



e// 



(2) 



neff{t) 



(10) 



Since &\ is in the form of a noisy linear measurement 
equation, compressed sensing decoding {i.e. £i-min recovery 
of Eq. 14 in |TT|) can be applied, even if m is smaller 
than n. However, robust ^i-min decoding requires the total 
measurement matrix, ^jot(f), to satisfy some conditions p2). 



|13|. Specifically, to ensure that the upper bound of Eq. 15 
in |11| holds, we have to investigate the satisfaction of RIP 



for ^tot{t), in our QNC scenario. In |11|, we proposed an 



appropriate design for network coding coefficients, which 
resulted in improved delay-quality performance for our QNC, 
compared to conventional packet forwarding. In this paper, we 
analyze the satisfaction of RIP for "^totit), resulting from the 
proposed design of local network coding coefficients in | ,ll| 
(also described in section |lll|. 



III. Design of Network Coding Coefficients 

Matrices with good norm conservation property are shown 
to be good choices for measurement in compressed sensing 
| fT2| . RIP characterizes the norm conservation such that an 
mxn matrix, 0, is said to satisfy RIP of order k with constant 
Sk, if: 

|2 



l-<5. < 



\e-s\ 



< i + Sk, y sE 



k\\o<k. 



(11) 



Random matrices with Independently and Identically Dis- 
tributed (i.i.d.) zero mean Gaussian entries are proved to 
satisfy RIP with an overwhelming probability [14|. Explicitly, 
for an m X n matrix with i.i.d. zero mean Gaussian entries of 
variance — , called Gmxn, the probability of satisfying RIP of 
order k and constant 6k is exceeding 



(also referred as overwhelming) where 



m > Ki/i;log( 



k' 



(12) 



(13) 



and Ki,K2 only depend on Sk (theorem 5.2 in |14|). In the 



following, we mention a design (originally proposed in pTj) 
for the local network coding coefficients, Pe.e'it) and ae,v{t), 
which results in zero mean Gaussian entries for ^fot(i). 



Then in section IV we derive an upper probability bound 
for satisfying RIP in our QNC scenario with the proposed 
coefficients. 

Theorem 3.1: Consider a quantized network coding sce- 
nario, in which the network coding coefficients, ae,«(i) and 
Pe,e'{t), ^"6 such that: 

. ae,vit)=0, Vi>2, 

• ae,t)(2)'s are independent zero mean Gaussian random 
variables, 

• /3e.e'(i)'s are deterministic. 



For such a scenario, the entries of the resulting 'i'tot (t) ^e zero 
mean Gaussian random variables, and the entries of different 
columns of *fot(i), i.e. {'i'tot{t)}iv and {'i>tot{t)}i'v', where 
w,f' e V, V y^ v' , are independent. 

Proof: By choosing ae,v{t) =0, V i > 2, we have: 

-^{t)^ B{t)-F{t)---F{3)-A{2), (14) 

which implies that each entry of \I'(i)'s and also 'i'tot{t) 
is a linear combination of entries of A{2). Moreover, since 
entries of A{2) are zero mean Gaussian random variables, 
then the entries of ^(i)'s and also "^totit) are zero mean 
Gaussian random variables. Since entries in different columns 
of 'i'tot{t), are linear combinations of two independent sets 
of random variables, i.e. entries of A{2), then they are also 
independent. However, such conclusion can not be made for 
entries of the same column of 'i'tot{t). ■ 

IV. RIP Analysis and Tail Probability of ^2 -norms 

Satisfaction of RIP for random matrices is usually character- 
ized by its probability (or its lower probability bounds) [14J . 
Moreover, to approach the probabilistic satisfaction of RIP, 
we first need to derive an expression for the tail probability of 
£2 norms |14|, |15J. Specifically, a well behaved "^totit) (i-^- 
"^totit) with high RIP probabiUty) should be such that 



P( ll*totW 



1 



>e 



(15) 



is very small, for all x, with \\x\\2 — 1- In the following, 
we calculate this tail probability for our QNC scenario with 
the proposed network coding coefficients, and then present 
a theorem which explicitly describes the relation between the 
satisfaction of RIP and the tail probability of Eq. [15] In the rest 
of this section, we assume that the conditions of Theorem |3.1| 
hold. 

Consider 

z! ^^tot{t)-x, (16) 

where x £ M", and ||a;|L = 1. Since the conditions of 



Theorem 3.1 are satisfied, Eq. 14 holds, and therefore: 

*f2) 



*tot(i) 



*W 



r!(t).A(2), 



(17) 



where 



n{t) = 



B{2) 
Bi3)F{3) 



(18) 



This implies: 
or equivalently: 



(19) 



B{t)F{t) ■ ■ ■ F{3) 
i = VL(t) A(2) ■ X, 

n 

V = l 

n \£\ 
^^{r!(t)},e {^(2)}e. X,. (20) 



By expanding z'^, and using the fact that {yl(2)}ey is non-zero 
only when tail{e) = v, we have: 



-,'2 



(21) 



where: 



7e 



\£\ \£\ 

E E le,e'{x){A{2)}e^tail(e){A{2)}e,^tailie'), 
e=l e' = l 



'{x) = ^{il{t)}ie {n{t)}ie' ■ Xtatl(e) Xtatl(e') (22) 



Using eigen-decomposition, (21 1 simplifies to: 

\£\ 



I- II2 — 7 ,^e(g) 



■Xe 



(23) 



where Ae(a;)'s are eigen-values of the symmetric matrix 

r(x) = [7e^e'(^)]|£|x|£|, (24) 

and Xe's are independent Chi-Square random variables of first 
order. 

Moreover, for the characteristic function of ||2'||2, we have: 






Iln 



JujXc{x)x^ 



e=l 
\£\ 



nird 



1 V"! -j2ujXeix)' 



(25) 
(26) 

(27) 



where (26 1 is derived from independence of Xg's. By using the 



inverse formula of characteristic function, Eqs. 28p0 can be 
obtained, where P|u,||2(.) is the probability density function 



of ||z'||2, and (29 1 is resulted from the integral property of the 



Fourier transform. The right hand side of ( 30 1 is the expression 
for the tail probability of ^2-norms, for a specific x, resulting 
from our proposed network coding coefficients. 



In the following, we present Theorem 4.1 which clarifies 



the relation between the tail probability of (15i and the 
probability of satisfying RIP /or a general case. 
Theorem 4.1: Consider $ for which we have: 



Pta<;(*:e) = maxP 



|$-x| 



1 



> e 



S.t. ||X||2 = 1 



(31) 



In such case, for every orthonormal 0, 8 = $ • satisfies RIP 
of order k and constant (5^, with a probability exceeding. 



p^jp $,fc,4 =1- 



(f)'p,„,(*..^^)^ 



(32) 

Proof: [^ To prove that RIP holds, we should show that 
inequality of ( [TT) is satisfied, for all fc-sparse vectors, s. We 
only need to show it is satisfied, for vectors, with ||s||2 = 

'Most of the proof is similar to the proof of Theorem 7.3 in |l5|. 



Ik'll2-i 



>e = 



1 



l-c 



1+/ PlI^'II^M^'^^ / P\\z'\\li'')di' 



OO 

1 /•+°°E[e^'^ll^ 



271" J -co 

TT / ,„ ,.,rrl«l 



e^^"^ sin(ea;) 



1 r+^E[eJ^\\^\ 
27r y_oo -i^ 



— OO UJ 



ul% v^r~j2[jx:(xj 



duj, 



le-j-(i+.)rf^ 



(28) 

(29) 
(30) 



1, since ||<&0-s||2 is proportional with 



Now, fix a set 



T C {1, 2, . . . , n}, with |T| = fc, and let Tt be the subspace 
of fc-dimensional vectors, s^, spanned by columns of $, with 
indexes in T. According to lemma 7.5 in |T5|, we can choose 
a finite set of vectors, Wr^ g Wt, where Wt C Ft and 
\\ui\\2 ^ 1' such that for all Sy G Tt, with ||sj^||2 < 1, we 
have: 



W' 



■Til 2 



< 



14' 



conditioned on: 



I^T|<(|^)^ 

Ok 



(33) 



(34) 



There are 



k 



different T's, for which we repeat the above 



procedure and obtain: 



W^IJWt. 



By using the union bound and the fact that for every x = (j)-'w. 



where w E W, Eq. 31 implies: 



1$ • x\ 



1 



>e <p,„(($,e). 



NI2<II<&0-M||2<(1 



Therefore, for every w G W, the inequality 

Sk 
1-^ 

holds with a probability exceeding 

.42 



4 s|| ||2 
^)INl2, 



1 



(^) Pt«i(*, 



71^ 



(35) 



(36) 



(37) 



The rest of the proof uses the same reasoning procedure, as 
in the proof of Theorem 7.3 in 1 15 1. ■ 

It can be concluded from Theorem 14.11 that in order to 
have a good RIP satisfaction {i.e. high upper probability 
bound for satisfaction of RIP), a small worst case tail prob- 
ability, Ptai; (■;"%)' is required. In section V we compare 
Ptaiii'j "%)'s> corresponding to our designed "^totit) and i.i.d. 
Gaussian matrix, to numerically evaluate their RIP behaviour 



By using the derived tail probability of ( 30 1, and applying 



Theorem 4.1 the following theorem can be obtained, which 
suggests an upper probability bound on the satisfaction of RIP, 
in our QNC scenario. 

Theorem 4.2: For a quantized network coding scenario, in 
which the network coding coefficients hold the conditions of 



Theorem 3.1 for every orthonormal (j), the resulting 8 



^tot{t) ■ (f) satisfies RIP of order fc, and constant 5k, with a 
probability exceeding 



PRIp['^tot{t),k,5^ 



defined in Eq. 38 

It is however difficult to derive the number of required 
measurements, m, from the expression of Eq. l38j we use 
numerical evaluations, in section W\ to explore the properties 
of our QNC design. 

V. Numerical Evaluations and Discussion 

In order to evaluate the RIP satisfaction of $fot(i), resulting 
from the proposed network coding coefficients, we use the 
worst case tail probability, Ptaii(v^)- This is because of 
the deterministic (linear) relation between PtQi;(., -fs) and the 
proposed upper probability bound in Theorem |4.l| Moreover, 
we calculate the worst case tail probability, corresponding to 
an i.i.d. Gaussian matrix, called G,nxn, and compare it with 
that of our 'i'tot{t)- For an m x n i.i.d. Gaussian matrix, 
Gmxn, the worst case tail probabiUty, Ptaiii^' 
be calculated as: |3 



Ik. 



■ tail 



{Gr, 



Sk . 
V2' 



1 
1-- 

TT 



-J" sin(a; * 



V2' 



c.(l-2j-i^)W2 



duj. (39) 



To present our numerical evaluations, for each value of 
tail probability, represented by Pj^ij/, the minimum number 
of required measurements in 'i'tot{t), resulting from our QNC 
scenario (with the designed network coding coefficients, as 
in Theorem 3.1 1 and G„ixn, are calculated. This is done by 



generating random deployments of networks and calculating 



the worst case tail probability of ( 30 1 and ( 39 1 in each 
generated deployment. The resulting tail probabilities, and 
corresponding number of measurements are then averaged 
over different realizations of network deployments. In Fig. [T] 
Ptciii is drawn versus m in logarithmic scale, for '$tot{t) 
(QNC) and Gmxn (Gaussian), and different values of RIP 
constant, Sk (Sk = 0.41421 ~ ^/(2) - 1 is the largest RIP 
constant for which Theorem 4.1 of fill can be applied). 

The statistical characteristics of the resulting "^totit) and 
its worst case tail probability vary by changing the network 
deployment parameters, like the distribution of edges in the 
network. In Figs. 1(a) to 1(c) the curves correspond to 
different deployments with n = 100 nodes, and |£| = 

^This can be obtained similar to the reasoning procedure for Eq. 30 
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Fig. 1. Logarithmic tail probability versus logarithmic ratio of minimum required number of measurements in our QNC scenario and i.i.d. Gaussian 
measurement matrices, for n = 100, different RIP constants, and different number of edges . 



1100,1400,1800 uniformly distributed edges, respectively. 
To generate the network coding coefficients, ae,j(i)'s and 



/3e,e'(i)'s, we make sure that the conditions of Theorem 3.1 
are satisfied. Moreover, for /3e,e'(i)'s, it was experimentally 
understood that the resulting 'i'totit) has a better behavior in 
terms of RIP satisfaction (and also ^i-min recovery) if in any 
two outgoing edges, /3e g/'s are orthogonal. 

By studying the curves in Fig. [T] the following arguments 
can be made: 

• The minimum number of required measurements for 
^tot{t) to achieve a worst case tail probability as a 



perfect i.i.d. Gaussian measurement matrix is in the same 
order as that of i.i.d. Gaussian (the logarithmic difference 
between the number of measurements for QNC and 
Gaussian cases is less than 1). Therefore, the number of 
required measurements in our QNC, for an overwhelming 
probability of RIP satisfaction (Eq. 12 1 is in the same 



order as that of an i.i.d. Gaussian matrix. Furthermore, 
this behavior is improved when the number of edges in 
the network increases, or the corresponding RIP constant 
is increased. 
By applying Theorem 



4.1 on the resulting P(£,i/> the 



PRip{'^totit),k,Sk) =1- 



i-r 

5k 



1 min 

TT X, ||2ll2 = l 



^■'^ sin( 



73^ 



nlfji Vl-j2c^Ae(x) 



zduj 



(38) 



lower bound on the RIP satisfaction for each sparsity, 
fc, can be obtained. Therefore, as an implication of 
RIP (Theorem 4.1 in 111]), we can make the following 
probabilistic statement about £i-min recovery error, in 
QNC scenario: 

Consider the QNC scenario, described in Theorem 4.1 of 
477V, in which we transmit k-sparse messages. In such a 
scenario, if the resulting '^tot it) corresponds to a point 
with Pf.^j^i{'^tot{t), ^) on one of the evaluated curves 
of Fig. n\ then the i2-norm of recovery error, using the 
£i-min decoder of Eq. 14 in [111, is upper bounded 
according to (15) in ftll^, with a probability exceeding 

PRIp{^tot{t),2k,S2k). 

By calculating the lower bound for RIP satisfaction (using 
1 1, corresponding to one of the points on the curves 
T] it would be clear that the possible sparsity, k, for 
which the resulting pj^jp{'$tot{t), 2k, 52k) approaches 1, 
is very small. In other words, QNC requires a lot of 
measurements to guarantee the upper bound of (15) in 
pTj , with an overwhelming probability. However, this is 
also the case for i.i.d. Gaussian matrices, as it has been 
previously pointed out by the authors of [16J-|,18J, that 
the RIP analysis for i.i.d Gaussian matrices proposes an 
exaggerated minimum number of measurements, required 
for robust ^i-min recovery. In conclusion, the minimum 
number of measurements, required for guaranteeing ro- 
bust ii-mm decoding, using our proposed '^tot{t), is in 
the same order as that of i.i.d. Gaussian matrix. The 
aforementioned fact (on exaggerated required number of 
measurements) can be considered as a weakness of RIP 
analysis, used in the compressed sensing literature. 

VI. Conclusions 

Joint distributed source coding and network coding of 
sparse messages with compressed sensing perspective was 
discussed in this paper. We investigated the satisfaction of 
RIP, in a modified random linear network coding scenario, 
called quantized network coding. This was explicitly done by 
using mathematical derivation for the tail probability of the 
resulting measurement matrix in our QNC scenario, and that 
of i.i.d. Gaussian matrix. It was numerically shown that our 
linear measurements have the same RIP behavior (in terms of 
order of minimum number of required measurements) as i.i.d. 
Gaussian measurements. Our RIP analysis provided us with 
the preliminaries for guaranteeing robust ^i-min decoding, in 
QNC scenario. 
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